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ABSTRACT 

Because discre4)ancies between individu'al needs and 
institutiornal roles are so bf ten ^ni&glected in pJLanning educational 
change^ attention .;to^ 'this .dimensioh may prove * worthwh ile . ^valuators 
who have omitted the e*valua<:ibn of th.e effects of educational 
innovation on the total system — not just the students'; — may find it 
productive* to assume this additiona,l responsibility, it is - 
hypothesized that /greater productivity may -^.be .realized i£ . ''^ 
institutions undergoing change attend to an^ respond €6 ihe real . and ^ 
the projected' i^nxiet^ies of all those x4feponsi'6le fot.anS effected by. 
•the- change^ since implementors^ may undergo 'greater ch?inges than 
client's. Those - involved 'in effecting change may need assurance that 
they can be competent in their new roles, and that 'their needs will 
be identifTed a.nd responded to. Such identification' a nd supportive 
-response is a key to establisiiing a climate where change can be 
productive 'and "non-events," or busihess-aa-usual teaching Ainder the 
cloak of a project title or number, yill be at a- minimum. The 
transactionally orient'ed evaluator qould pa»rticipate in', the usual 
f ormati ve summat ive and cost tasks^ but'he would also ask additional 
qtiestions about (1) ^Who is* involved? (2) What is expected of them? 
(3) .How are they threatened by change? (U) How do they make 
compromises between their need^ and the reward system? (5) In what 
way, and how adecjuately does the institution assess the consequences 
of change., not only on the system clients, but alscT^on the members of 
the system? (A.uthor/BJG) ^ ' . * . , 



Documents, acquired by ERIC Miclude ma^iy informal unpublished 
materials not available from other sources. EPIC ma^es every effort 
to obtain the^ best copy availa-ble. ^Nevertheless , it ems of marginal 
reproducibility are often encoun-ter^d and. this^ffects the quality 
of the micro'fiche and har(|copy' reproductions . ERIC makes available 
vid the "ERIC DoQifment Reproduction. Service (EDRS)* EDRS is not 
te^onsLbl.e for, the quality ,o5 the original documents Reproductions * 
* supplied by EDRS are the best t^h^t can be made from th6 originals * 

**i«(******:)e****4r************** ************* 4c** ************* **^* ********** 



. April L973 

,^ • • TRANSACTIONAL EVALUATION AND 

^ . ' ' THE IMPRO\^>ENT OF INSTRUCTION 

' I- 
Robert M. Rippey, Ph.D. 
The University of Connecticut 

Health 'Center ' , ' ' 

Farmington, Connecticut 06032 

In the quest for the answer to the question, "why is change iiV school,s 

^ ■ 

so difficult?", I have become involved in a number of evaluations ranging in 
froa single classrooms through large segments of a very large school system. 
In the evaluation of three large scale efforts at inter-institutional colla- 
boration among a university, a school system, and a community; 1) The^Woodlawn 
ExperimentaL Schools Project, 2) The University of Illinois Teacher Corps, and 
' 3.) The Crane High School Satellite Program, I found *each of the involved tr. • 

institutions stressed by the vicissitudes of change. One thing became clear 

t ^- . . 

about each of tH^se afttempts * t;o move an unw'ieldy insi^feution. The teac^hers 

« vere usually called upon to change more than the students. ^ 

Although traditionally much effort is spent studying the consequences 

of new programs on students, the traditional approach to evaluation often 

. spends precious little 'time studying what happens to the teacher and oth6r 

personnel in the school. As a result, ev'aluations have often been made of 

, *• • 

\ " ' * 

programs which were in fact not implemented, or poqrly implemented at best. 

Acknowledgment of the fact led the federal government to demand evaluation of 

the degree of^'fipplementation- in Project Follow Through, and Charters (1973) to 

issue a statem'ent of cor^ern over the possibility of evaluating "non-even^ts. I' 

Although concern for non-events i3 a welcg^e change for the evaluator 

• ■ / 

wpOj^ih thew past, has often- been content to plod aloug collecting his summative 

/'data while the ship was siilking, it is possible for the* evaluaXor to do better*. 

* / - - ' . 

On a strictly objectives basis, using a spectrum tools from behavioral 

sciences, he can learn why teachers and schools encounter such difficultly when 

the demands of a new prpgram are articulated. Secondly, he can share this 

2 



» • * . • ' 

• inforaaci^on with p'roject directors and staffs' as a p,art of fqrriacive evaluation 

This infoirnation, acknowledged and responded to can ultimately reduce the 
frequency ''of non-events. ^ 

There are many forces which 'af^ecc change in schools. * Corwin (1973)^ 

• . . . ' ' 

in his scudy of che b;acional Teacher Corps suggesced the f o^LiLowing : 1) Power 

Structures, 2) Resources, 3)' Task Structures, A) Status Cjiaracteristics of 
Members, 5) Occupational Environment. And of these,^ perhaps .the most impor- 
.^tant. is number four: the effect of change on the status structure or'system 
'or- roles played out by studen-ts, parents, teachers, and administrators and 
the threats to these roles which change presents. These perturbations of 
status structures are -a primary target of transactional evaluation. 

Autonomy and isolation are two primaryjcharacteristics ,of the teacher.'s 
work situation. A teacher may value his/her autonomy highly. He may ever^ be 
willing to strike for it. Yet the autonomy can be tied to isolation. How- 
eyer, the teacher, as captain of his classroom, ^isolateed spatially by his . 
four walls and .temporaly by his busy schedule, may still not be the master 
of his soui; For isolation, which is the price- he may have to'^pay for his 
autonomy, may shield him from the inputs he needs to plan well, and may wall 
in his outputs to such an extent that he cannot' be influential in effecting 
change. Although autonomy may be prized for its own sake, the concoinraitent ' 
isolation may lead to alienation and perhaps to a' sense of powerlessness and 
despair. Thus the origin of such phrases as "They won*t let us do it," "No 
on^ cares." Is there any way to reconcile an individual's needs to assert his 
identity and aspirations with the sometimes implacable and contradictory 
expertations of his institutions? /V 

\ '' ' 

At different times in history, different answer^ to these questions , 
have been popular. In those places and times where hunger was common,, human 
life cheap, 'and despair universal, the answer was "no". Both man' and 

3 



his mscitutions were^perceived as so weak- and imperfect that there was no 
'hope. This position is the essence of sone of the eastern religions. A 

second position assumed that institutions were adequate but people were not. 

It was the task of the institution to make individuals confoVm. When. I was 
in the Coast Guard, the rule was "shape up or ship .out." A third position 
held that the institution should adapt to individual demands and needs. 
Rousseau popularized the third position. All of these three positions, \ 
though placed in the order of appeal, have serious defect^. The deficits of 
the first two positions are "obvious and Hogan (1973) has eloquently laid to 
rest the pretensions of the various schools of individualism. • The fourth 
positioa is the transactional position'; It considers both the individual 
and the institutions powerful and necessary. Ari individual's identity is 
related to the institutional role he plays. Nevertheless, institutions must 
be tesponsive if the^y are to contribute to the quality of individual Uves. 

♦ 

Individuals need to understand h9w instritu&ion make demands on them and 
affect their lives and their character. The individual, in turn, has res- ' 
ponsibility to make demands-on his systems; *to engage in many transcation^ 
with them in order to make certain nefeds are met. But how c^n these trans^ 
actions between individual and^ institutions take place? In any group or 
institution resources need to be committee to maintaining negotiations 

"Xcross the, transactional gap between individual and institution, 

_ A classical model \)f organizations attributed to Getzels and Cuba 

^ (1957) consists of an institutional dimension made up of a set of roles, 
norms, and rewards and sanctions placed in^opposition to an indiyidual dimen- 
sioa-of 'personality characteristics and needs. ' ' - 
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MODIFIED GET2ELS-GUBA MODEL OF AN INSTITcTION 

Institutional Rfele/ Norms and Expectations Rewards Sf^ 

t ' ^ , and 



Transactions 



Sanction^' 1 •^^'^^ ^ 



Satis- I — Goals 
/ faction"^ ^ — ^ 



^' Individual Ideatity Personality Characteristics Nee^s -B^^^^^ 

Such a model can show the relationship of transactional evaluation 
to foraative, sunniative, and cost effectiveness evaluations. Summative evalu- 
ations generally evaluate- program effectiveness in terms of goals. Cost 
effectiveness studies 'deal with a'limited aspect /6f the efficiency dimension. 
Transactional. evaluation is an elaboration of Ebrmative evaluation which is 
concerned vitn the satisfaction dimension and the other vertical lines between 
tne institutional dimension and the individual 'dimension. Perhaps satisfaction 
is a bad term to use since it has so many emotional connotations. Perhaps it 
denies the beneficial constructive tensions and conflicts which characterize 
vital institutions. That denial is not intended. Transactional evaluation 
looks at institutional roles and as^s, "What are the roles defined by the 
institution?'*' "What is expected?" "What will happen to me if I don't conform?" 
It a\so asks, "i;ho are the people who are to fulfill these roles?" "What do 
they need to develop coramitment?" Finally, "What are the conflicts if any, 
between the institutionally an^ individual dimensions?" It also asks questions 
aUout how people manage to, deal with the strains that ari^e due to proposed 
and implemented "changes. Are the normal formal and informal aspects of' the 
institution adequate to deal with the strains which change imposes, or are 
additional institutional or personnel changed needed in order to maintain 
* performance and progress toward old and new goals? 
* The transactiortally oriented evaluator could participate in the 

usual formative, summative and cost tasks, but he^would also ask additional 
EBsJC questions about D Who is involved?, 2) What is expected of them?, 3) How 



ore they threate'ned by change?,- 4) How go they make corap.ro^ises between 

♦ * • - 

:hrir needs and ti^e reward systen?, 5) In what way, and how adequately does 

:r.e institution assess the' consequences of chagne, not only on the system 

clients, • but also ©n the members pf the system? 

Organizations such as assembly lines .turniag out a constant product 

» tea stable ziarket require little by way of transactional activity. Roles 

are clear and ^people accept positions knowing what is expected frob them and 

♦ * 

vnac they can 6>rpect: from the job/ On the other hand, wh&re demands from 
thoth outside and inside an instijiution lead to changing roles and expecta- 
cions, transaction is at a prenium. 

Unfortunately, these transaction's do not take place automatically, 
:<=t2 6 Kahn (19o6) have identified organizational subsystem which facilitate 
: ransactions. These subsystems occur to a greater or lesser degree' in different 
organizations and are devotej^ to maintenance of the work structure, obtaining 
i -istitutional support, mediating between employee needs and institutional 
needs, and adaptation to the pressure of change. These institutional sul^- 
systems may or may not be prominent in schools. Often their' functions are 
performed out or dire need, informally, in the teacher^s lounge over the noon 
hour. This informal performance of crucial functions is adequate when the 
demands on a school are small. But when the heat is on, love is not enough. 
Without either the time or the expertise, schools subjected to intense demands 
for change often defend thems/lves by becoming more rigid, by isolating them-f 
selves, by denying the existence of problems, by rejecting the invading demand 
^ or idea for change or somellimes, by simply falling'ap^rt. In order to avoid 
overreaction to change^ Lt is necessary to build a climate for change. A 
favorable climate for clyange is nurtured by starting change on a si^ll scale, — 

on an experimental basys'. In addition to* this, it is necessary th^t the 

* / • 

^ functions of maintenance and adaptation be performed adequately. Transac- 

ERIC , G 

ummvj^mm tional evaluation bedbmes a necessary ingredient* ^ 



♦ 

i 

The need for transactional evaluation in times ()f institutional 
stress has been exemplified recently in a study of the' human' impact of the 
>k:nagua earthquake, (Kates, Hass, Amaral, Olson, Ramos, ^nd Olson, 1973). 
Although^ if is unlikely. to find a schopl in^ as dire stress as a city recently 
leveled by an earthquake, the analogue is not unreasonable,' Under the extreme* 
sti:assc-a£_ the earthquake the normal human reactions to stress- were amplifie^. 
Kates reports that the very first activity after the earthquake occurred was 
a kind of evaluation: "Initial assessment of physical -AND human effects: 
thrgugh direct observation^--contactiiig^thers , seeking to contact others, 
seeking to discover what has happened, who is hurt, and who is safe*' (Kates^ ' 
et al, 1973). The second activity involved immediate efforts to ensure indi- 
vidual survival. The third activity reported again ^involved a kind of evalua- 
tion - at least information gathering - efforts to search for the trapped and ^ 
injured. ^Once information was ga^thered, efforts we ce -made to establish , com- 
□unicatio;i3 and th$ stimulate the flow of information. Although I have cite'd 
an extreme case, this response to'an earthquake begins to look like what I 
refer to as "transactional evaluation." For the fundamental untrained response 
to the earthquake was an immediate assessment of the gap between i-ndividual 
needs and institutional capabilities and demands, followed by the stimulation 
"of information flow, 

^Elafjoration of basic concap-ts ^nd case studies may be found in 
Studies in Transactiona l Evaluation . (Rippey, 1973), In brief, transactional 
evaluation attempts to uncover the apprehensions of persons involved in insti- 
tutional change, whether the change be an earthquake .or a computer terminal 
in a^classroom. Its techniques can be used at all levels from the classroom 
teacher to the top administrator. It involves institutional introspection, 
Whenever someone asks. "How is this change affecting the feelings of the 
people either in or involved with this school," he is interested in trans- 
actional evaluation. *y 



Tran3acti.onal evaluation is a ccntinuous process which takes place 

. * ' / 

in four phases, 1) Identificatioo of transactional issues, 2) Consensus 
testing and priority setting, 3) Fact finding, 4) Decision making and action. 
Tne cycl^ may then be repeated at regular intervals. 
Icent^if icat ion of Transactional Is sues 

The first step of the cycle consists of exploring the concerns of 
persons involved by the change. It is important to sample a variety of share- 
Holders. Parents, teachers, and students, are a necessary but not- always 
suificient set of interest groups whose roles and , needs should be explored. 
Sampling of these concerns may be carried out by means of what I refer to as 
a transactional questionnaire. 

Perhaps the most important characteristic of a transactional ques- 
^ cionaa/re is that the items are solicited from as many persons representing 
each identified interest group as is feasible. These statements are then 
-lilized with a qinimum of editing, and selection. Of course redundant items 
^re screened out an^a occasional rewording may be necessary. But in general, 
the statements should be kept as. close to the original wording as possible. 
You mght ask, "Should ambiguous statements be included?" I know jol no 
theoretical answer to this question but in practice an ambiguous statement* 
nay precisely represent the state of mind of a particular group and therefore 
flight be eminently suitable for identifying that issue. 

It should, be clear then that such instruments can only be developed, 
for reasonably sized groups lest, the instrument itself become unwieldly. 
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FIGURE 1 








Transactional lEvaluction Instrument 
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•Roles: ?/C = Parents and/or Community; Stvi = Students; 
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Items of high quality can be stimulated by ^ few good initial probes, or 
even by some role playing on critical issues prior to the solicitation of 
items. Items joay also be contributed to the questiqnnaire by the evaluato-r 
in case no one mentions the usual concerns such as: not certain I will 

.be able to do a good job in my new role" or "1 wonder how the parents will 
react to the new program?" 



Concensus Testing and^Priority Setting 

Once the items have been assembled into a questionnaire, it is 
administered to the persons who contributed the items. The results are 
su^ziarized and the meaning of the items, the .implications , of the new infor- 
Elation, andthe appropriate consequences are discussed. In some instances the 
picture IS clear and in other cases further information is necessary. When 
new information is needed, this suggests the developmentjDf an evaluation 
connittee which will utilize^ a wide range or research and evaluation liechniques 
to answer the crucial questions raised in the initial issue identi€ida4:ion • 
Further Inforgiation Gath'fering ^ ^ ^ 

An evaluation committee may be crucial part of the transactional 
e\^aluation process. Although .every educational evaluation will probably have 
a fo^raativ^ and suramative avaliiation plan outlined prior to implementation or 
funding, the transactional evaluation plari must grow along with the program. . ' 
The evaluation committee, made up of both protagonists and reasonable, anta- 
gonists will bJ r^ponsible for planning > and implementing or having impJ^e- 
mented, the design which will be governed by the primary concerns of each 
interest group. In- the performance of this task, the evaluation committee 
•will employ a wide range of research and evaluation skills. It may attack 
a 'variety of questions not commonly encounte^red in a more conventional evalu- 
ation. For example, in the Crane High School 'program, some of the questions'- 
explored were:;l) Would parents of more able students really sen4 their. children 
to Crane if a more demanding academic program should .be offered? 2) Would the 

V 

board of education be willing to make certain modifications in their design for. 
remodeling the building in, order to accommodate proposed ehanges and 3) In view 
^ of recent moves in opposition to testing, how would the community react to an 
increased emphasis on testing within the high school? ^ 

o 10 • ~ 
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Decision Making and Action , 

Once full information is available and priop.ties are set through 
group discussion, adjustments to the program plan can be made. This adjust- 
• ment then leads to another recycling through the process. In a stud^ of the 
impact of computer terminals in the classroom, Compton (1975> and Compton et. al, 
(1975) found the following procedure useful and informative. 'In preparing 
a new instrument, they asked the participants to review. the old items and 
classify them as to w^jiether they continue to be issues. If 10% or more 
considered that an item was still an issue it was placed' on the new instru- 
ment.* If not, the item was classified as resolved. Participants were also 
asked to submit new , items which were tAen incorporated into the questionnaire. 
He thus obtains a profile over time of the onset, development, and resolution 
of crucial issues. This profile can then be associated with the chronology 
bf, the program. ' * 

Psychometric Issues 

Up to this poi,nt little has been said about the psychometric issues ' 
of reliability, validity, and generalizability . The relevance of the^e issues 
depends on thS^hase of the evaluat-ion. 

The transactional instrument itself perhaps requires anti-reliability, 
and anti-generalizability. Since the purpose of . the instrument is to explore 
everyone's concerns', not just concensus, the instrument will exploit rather than 
be impeded by a large amount of subject-item] interaction. Furthermore, if the 
instrument is comprehensive it will probably lack any semblance of item homogen- 
eity^. Such characteristics do not, usually contribute to high reliability coef- 
ficients. However, high reliability coefficients are of no value at this point 
of exploration and identification of issues. ' 

' il 
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Nor does generalizability seem to be a particulary useful charac- 
teristic for^the first stage of transactional evaluation since transactional 
evaluations aim is specificity. The transactional instrument is prepared 
for a particular group of people, .here, and now. The same set of items will 
probably never be repeated for a different group ^in a different setting, nor 
should it. ^ 

On the other hand, validity is always important* At least, the ' 
meaning of responses must be clear., This is done through*^ the discussions 
in phase two, and through the further information gathering of phase three 
^hich will confirm and elaborage upon the meaning of the initial responses. 
At this- point the entire range of psychometric skills should be employed. 
Su'imary 

Because discrepancies between individual needs and institutional 
roles are so often neglected in planning educational change, attention to this 
dimension may prove worthwhile. Evaluators whohave ommitted the evaluation 
of the effects of educational innovation on the total system — not just tl^ 
students — may fnnd it productive to assume this additional responsibility. 
Further research on transactional 'evaluation should test the hypothesis 
that greater productivity may be realized if institutions undergoing change 
attend to and respond to the real and the projected anxieties of all those 
responsible for and affected by the change. For implementors may undergo 
greater changes than clients. ^ Those involved in effecting change ma}{ 
therefore need more assurancie that they can be competent in their new, roles, 
and_ibat their needs will be identified and responded\^* Such identification 
and supportive response is a key to establishing a climate where change can 
be productive and "non-events" will be at a minimum (Charters, 1973) • 1a 



^ ,^^^non-^-event, accbf^ing the Charters is. b{jsiii<3ss-as-usual teaching under the ^ 
. ^cloak of a project title or'n^mb^r. Transactional evaluation can help idesitify 

' ' ^ * ^ ^ ! 

» \eeds whi(;h are of tea felt but seldom admitted because of their ink)lication 

. ofjersoaal weakness. ' ' " ' • 

• . . *' 

.The res^)onsible internal, evalu^tor will see to it 'that th^ project ^ 
he^is evaluating dcfts not fail ^for lack of insight into the impediments to ' ^* 
impl-era^ntatioA whic'h he is in a 'unique position to explore, aA evaluation ' • 
-»^N v'l^resents an incon^^iUte picture out of* context 'if it does not explore and 
descifi^ the perturbations ofxthe system undergoing change. Having access 
bo Such inforptioa, the evaluator is negligent if l^e does not feed it back. 
' ^If no one looks at 'the impediments, it is .likely that teachers will begin to 

.; ' " • ^ * 

sp,end r.ore. afH more time constructing non-events which give the illusion of 
^r. progress.' This wheel spinning may be functional to them in preserving *t:he 
; asi?ects of tl[iei< jobs with which th^y feel Eost comfortable.' Thus non-Qvent^'^ . 

-.may subvert program design-'' whfle preserving the ^tltus quo under^an illusory 

' r / ^ ^ \ 

cqver^Qf superficial change. How many innovations \eserve, the epitaph 

^"rt was. a -good-l'i^de^, ^"^^^ really did it?"' did it?", 
^ > ' Jk ' ^ ^ * 
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